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ABSTRACT 

In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in NAEP ' s 
history, voluntary state-by-state assessments were made. In 1992, the NAEP 
Program included an expanded TSA in fourth-grade reading. The 1992 reading 
assessment considered students' performance in situations that involved 
reading different kinds of materials for different purposes. The fourth-grade 
assessment measured two global purposes for reading- -reading for literary 
experience and reading to gain information. In Iowa, 2,756 students in 129 
public schools were assessed. This report describes the reading proficiency 
of Iowa fourth-graders and compares their overall performance to students in 
the Central region of the United States and the nation (using data from the 
NAEP national assessments) . The distribution of reading results and reading 
achievement level results are provided for subpopulations of students 
(race/ethnicity, type of location, parents' educational level, and gender, 
and performance according to purpose for reading) . To provide a context for 
the assessment data, participating public school students, their reading 
teachers, and principals completed questionnaires which focused on: policies 
and practices related to reading (time for instructional activities and 
instructional resources for reading) ; delivery of reading instruction 
(instructional materials and activities, workbooks /worksheets /writing, 
discussion and group activities, time to read, reading and use of libraries, 
assessing progress in reading) ; reading instructors (preparation, experience 
and professional development); students' home support for literacy (reading 
outside of school and in the home and hours of television watched per day) . 
The average reading proficie ncy o f fourth-grade public school students in 
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Trial State Assessment 



The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation’s only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientifically selected samples of youths 
attending both public and private schools and enrolled in grades four, eight, or twelve. The 
assessment questions are written around a framework prepared for each content area -- 
reading, writing, mathematics, science, and others — that represents the consensus of groups 
of curriculum experts, educators, members of the general public, and user groups on what 
should be covered on such an assessment. Reporting includes means and distributions of 
scores, as well as more descriptive information about the meaning of the data. 



New Reading Assessment Framework and Questions 



The goal of the National Center for Education Statistics is to make data available for the 
public and to do so in accurate and understandable ways that are not misleading. The task 
is challenging because much of what matters in NAEP is changing: 



• the content in response to the developing standards of various curricular 
groups; 

• the assessment questions in response to new developments in assessments; 
and 

• the reporting in response to increasing interest in student achievement 
relative to standards of student performance. 
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The framework for the 1992 Trial State Assessment Program in reading considered 
students’ performance in situations that involved reading different kinds of materials for 
different purposes. The fourth-grade reading assessment measured two global purposes for 
reading - reading for literary experience and reading to gain information. (The eighth- and 
twelfth-grade national NAEP reading assessments also measured a third purpose for 
reading -- reading to perform a task.) Reading for literary experience usually involves the 
reading of novels, short stories, plays, and essays. In these reading situations, the reader 
can determine how the author explores or uncovers experiences through the text and 
considers the interplay among events, emotions, and possibilities. Reading to gain 
information usually involves the reading of articles in magazines and newspapers, chapters 
in a textbook, entries in encyclopedias and catalogs, and entire books on particular topics. 
These reading situations call for different orientations to text from those in reading for 
literary experience because readers are specifically focused on acquiring information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

• Initial Understanding — comprehending the overall or general meaning of 
the selection. 

• Developing an Interpretation - extending the ideas in the text by making 
inferences and connections. 

• Personal Response -- making explicit connections between ideas in the text 
and a student’s own background knowledge and experiences. 

• Critical Stance — considering how the author crafted a text. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. 

The 1992 NAEP reading assessment uses a variety of innovative assessment approaches 
that are considered significant advancements over previous assessments. In addition to 
multiple- choice questions, the assessment primarily includes constructed-response 
questions that ask students to demonstrate comprehension beyond a surface level. Also, 
longer and naturally-occurring reading materials are used to provide more realistic reading 
experiences than in previous assessments. 

Taken together, the changes in the 1992 reading framework and assessment activities 
preclude any comparisons between the results in this report and those for previous NAEP 
reading assessments. If the current NAEP framework is used in the future, as planned in 
the 1994 assessment, the 1992 reading data will supply the basis for a trend report. 
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A Transition in Reporting 

Over time there have been many changes in emphasis of NAEP reporting, both to take 
advantage of new technologies and to reflect changing trends in education. In 1984, a new 
technology called Item Response Theory (IRT) made it possible to create “scale scores” 
for NAEP similar to those the public was accustomed to seeing for the annual Scholastic 
Aptitude Test (SAT). Educational Testing Service, in its role as Government grantee 
carrying out NAEP operations, devised a new way to describe performance against this 
scale, called “anchor levels.” Starting in 1984, NAEP results were reported by “anchor 
levels.” Anchor levels describe distributions of performance at selected points along the 
NAEP scale (i.e., standard deviation units). Anchor levels show how groups of students 
perform relative to each other, but not whether this performance is adequate. 

In 1988, Congress established the National Assessment Governing Board (NAGB), 
assigning it broad policy making authority over NAEP, including the authority to take 
“appropriate actions ... to improve the form and use of the National Assessment” and to 
identify “appropriate achievement goals for each . . . grade and subject area to be tested in 
the National Assessment.” To carry out its responsibilities, NAGB developed 
“achievement levels,” which are collective judgments about how students should perform 
relative to a body of content reflected in the NAEP frameworks. The result is translated 
onto ranges along the NAEP scale. For the 1992 reading assessment, this process was 
conducted for NAGB under contract by American College Testing (ACT), which has 
extensive experience in standard- setting in many fields. 

With this background, the initial reports for the 1992 reading assessment mark NCES’s 
continued attempt to shift to standards-based reporting of National Assessment statistics. 
The first transition to reporting NAEP results by achievement levels was for the NAEP 
1992 Trial State Assessment in mathematics. 1 The impetus for this transition lies in the 
belief that NAEP data will take on more meaning for the public if they show what 
proportion of our youth are able to meet judgmental standards of performance. 

Reporting NAEP results on the basis of achievement levels represents a significant change 
in practice for NCES. On occasion, this agency makes use of emerging analytical 
approaches that permit new, and sometimes controversial, analyses to be done. When 
doing so, this agency, just as other statistical agencies do when introducing new measures 
to supplement or replace old measures, also has provided the data according to the earlier 
procedures in addition to the new ones. In the case of the 1992 mathematics assessment, 
for example, the “anchor levels” or “scale anchoring” method of reporting was presented 
in an appendix. 



1 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993) and the individual 
1992 Mathematics State Reports. 
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In this assessment, the “scale anchoring” methodology used by NAEP since 1985 has been 
used but in a new way. As implemented for this report, the scale anchoring process applies 
not to regular scale intervals (standard deviation units), but to the achievement levels 
established for fourth-grade students. 2 The details of this procedure are presented in 
Appendix D. The critical distinction here is that setting achievement levels attempts to 
describe what students should be able to do in various ranges of the NAEP scale while the 
anchoring procedure attempts to describe what they can do at those achievement levels 
using actual student performance data from the NAEP assessments. 

Chapter 1 of this report describes how the 1992 standards were prepared and provides 
examples of assessment questions that illustrate the reading content reflected in the 
descriptions of the NAEP achievement levels. Chapters 1 - 6 include information on 
overall means, distributions of reading proficiency, as well as background questionnaire 
data, all taken directly from the results of the assessment questions. 

Continuing Development Effort 

We believe that the numerous completed and ongoing studies 3 will lead to national debate 
that can assure the public is well informed about these issues -- as informed they must be 
because the results will be a vital influence on what Americans come to think about the 
condition and progress of our schools. Indeed, measures of student learning may be as 
significant bases for public understanding about our nation’s education system as the 
Consumer Price Index and the monthly unemployment statistics are in informing the 
public about our nation’s economy. 

In addition, members of the public need the data in this report to see for themselves what 
standards-based reporting might do and to evaluate the often conflicting claims of adherents 
and detractors of these changes in approaches to reporting on the educational achievement 
of American students. Reporting NAEP results to the public would be more clear if the 
language of the achievement levels, or standards, could also directly describe what students 
know and can do. In order to accomplish that, the frameworks, assessment questions, and 
achievement levels may need to be developed in tandem. That is easier to say than to do, 
however, because it implies a substantially larger pool of assessment questions, carefully 
designed to support reporting about performance relative to a set of performance standards. 
Clearly this is a developmental effort that will take time and several iterations, during which 
data supporting appropriate inferences about the performance of American students will 
be gathered on a continuing basis. 



2 First, students were identified who performed at or around the three achievement levels on the scale (212, 243, 
and 275). Next, questions were identified that were answered correctly by 65 percent or more of the 
fourth-grade students at the cutpoint for that achievement level. Finally, reading educators were asked to 
analyze each anchor-level question and create summary descriptions of the skills and abilities evidenced by 
students who answered these sets of questions successfully. 



3 Educational Achievement Standards: Setting Achievement Levels for the Nation . The Second Report of the 
National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1992 Trial 
State Assessment. (Stanford, CA: National Academy of Education, 1993).; U.S. General Accounting Office 
(GAO). Educational Achievement Standards: N AG B's Approach Yields Misleading Interpretations. June 
1993. GAO/PEMD-93-12.; Assessing Student Achievement in the States. The First Report of the National 
Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1990 Trial State 
Assessment. (Stanford, CA: National Academy of Education, 1992).; R.L. Linn, D.M. Koretz, E.L. Baker, 
and L. Bur stein. The Validity and Credibility of the Achievement Levels for the 1990 National Assessment of 
Educational Progress in Mathematics. (Los Angeles, CA: Center for Research on Evaluation, Standards, and 
Student Testing, UCLA, June 1991) CRESST Report 330. 
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A Recent History of NAEP 

In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis. 

As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 4 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 5 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 

School and Student Participation in the Reading Assessment 

In Iowa, 129 public schools participated in the fourth-grade reading assessment. The 
weighted school participation rate was 100 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 100 percent of all the 
fourth-grade public-school students in Iowa. 



4 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement NAEP's 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

5 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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